Single Action Three-Dimensional Model Printing Methods

ABSTRACT

Methods and techniques of using 3D printers to create physical models from image data are discussed. Geometric representations of different physical models are described and complex data conversion processes that convert input image data into geometric representations compatible with third party 3D printers are disclosed. Printing templates are used to encapsulate complex geometric representations and complicated data conversion processes from users for fast and simple 3D physical model printing applications.

RELATED APPLICATIONS

The present application is a divisional application of U.S. patentapplication Ser. No. 14/036,620 filed on Sep. 25, 2013 (now U.S. Pat.No. 8,579,620 issued Nov. 12, 2013), which is a divisional applicationof U.S. patent application Ser. No. 13/039,048 filed on Mar. 2, 2011,both of which are hereby incorporated by reference in its entirety.

FIELD OF TECHNOLOGY

The present invention is in the technical field of three-dimensional(“3D”) printing and rapid prototyping. In particular, the presentinvention is in the technical field of 3D printing and rapid prototypingusing three or n-dimensional image data sets, such as CT (computerizedtomography) or MRI (magnetic resonance imaging) images.

BACKGROUND

Three-dimensional (“3D”) printing of physical models is useful in a widevariety of settings. Some potential uses include production ofanatomical bodies like bones for research and clinical applications,medical product development, machine design, and equipment design, toname just a few. 3D printing or rapid prototyping refers to a collectionof technologies for producing physical parts directly from digitaldescriptions. Digital descriptions include output of any software thatproduces a 3D digital model. One example of such software isComputer-Aided Design (CAD) software. Creating a 3D digital model from a3D image data set requires specialized imaging or CAD software. Rapidprototyping machines have been commercially available since the early1990's, the most popular versions of which build a desired structure byadding building material layer-by-layer based on a digitalthree-dimensional model of the structure.

However, because of the amount of user interaction time involved and thecomplexity of data conversion process between image data formats anddata formats supported by 3D printers or rapid prototyping machines,applications of the present technology of producing 3D physical modelsfrom three or n-dimensional images are rather limited.

FIG. 1 illustrates the current method of creating a physical model froman input image data set. The input image data set comes in the form of3D voxel data or serial, sequenced two-dimensional (“2D”) images. Avoxel (volumetric pixel or, more correctly, Volumetric Picture Element)is a volume element on a regular grid in a three dimensional space,having one or more numerical values as attributes such as intensity orcolor. This is analogous to a pixel (Picture Element), which has one ormore numerical values as attributes on a regular grid in a 2D image dataset. A 3D image data set may be organized as a series of 2D images and avoxel in a two-dimensional image plane may be referred to as a pixel.

In FIG. 1, when a user 11 needs to create a physical model 35 from animage data set 10, the user 11 looks up the image on his/her computer 15and transfers the image data 10 to an image processing operator 21. Theimage processing operator 21 loads the image data 10 set on his computer20 where special image modeling software is available. The imageprocessing operator 21 reads the instructions sent by the user 11 tounderstand what type of model is required. If the image processingoperator 21 still has questions or needs additional information, he willcommunicate with the user 11 to get the information. The imageprocessing operator 21 then starts the process to create a 3D digitalmodel 22 from the image data sets 10 on his computer 20 usingspecialized modeling software. The creation of the 3D digital model 22requires a trained operator 21, specialized imaging software, and asignificant amount of user interaction. The image processing operator 21needs to communicate frequently with the user 11 who has ordered thephysical model to understand the requirements and applications of themodel. The image processing operator is also required to spend asignificant amount of time to perform image segmentation and to tracemanually certain image areas. After the 3D digital model 22 has beencreated, it is then saved to a file format supported by a 3D printer orrapid prototyping machine 30, for example, the STL (stereolithography)file format. The digital model file is then sent to the 3D printer orrapid prototyping machine 30 to generate a 3D physical model 35. Thethree-dimensional (“3D”) printer 30 is likely located at a differentlocation and operated by a 3D printing operator 31. When the physicalmodel 35 is printed or fabricated, the 3D printing operator 31 sends itto the imaging processing operator 21 who then sends the finishedphysical model 35 back to the original user 11. The present 3D printingtechniques are complex and cost ineffective. The physical models maytake too long to create to be useful, for example, to an emergency-caredoctor.

As a particular example of the need for an efficient 3D physical modelprinting process, we consider 3D printing applications in the medicalfield. In a typical application of 3D printing techniques in the medicalfield, medical images are first ordered and acquired on a hospitalcomputer by a doctor. The doctor then sends the images to a trainedimage processing operator to create a digital model. The imageprocessing operator communicates with the doctor to understand therequirements for the model. The image processing operator loads theimage data set into a 3D image processing software to identify featuressuch as bones, tissues, etc. by using image segmentation software tools.Because image processing of medical data is complex and time-consuming,it remains a challenging task even to a professional image processingoperator.

After loading the image data, the image processing operator 21 thencreates a digital 3D model, for example, a 3D polygonal surface model byusing software-based modeling tools. As an example, one commerciallyavailable software solution, “3D-DOCTOR”, can be used to produce 3Ddigital models of anatomical structures, as described in Yecheng Wu,From CT Image to 3D Model, Advanced Imaging, August 2001, 20-23. Aftercreating the digital 3D model, the image processing operator 21 sendsthe digital model to a 3D printing service provider. The 3D printingoperator 30 at the 3D printing service provider loads the digital modeldata on his computer, controls the 3D printer to produce a physicalmodel, and then delivers the finished physical model to the doctor whoordered the model. The above-described process is user intensitive andrequires operators to possess advanced software training, knowledge ofthe intended applications, and a good understanding of the differencebetween image data formats and the various data formats supported by 3Dprinter and rapid prototyping machines.

In the above described process, one procedure employed in imageprocessing is image segmentation. Image segmentation refers to thedelineation and labeling of specific image regions in an image data setthat defines distinct structures. Image segmentation may include stepssuch as differentiating a particular structure from adjacent materialhaving different composition and identifying distinct objects having thesame or similar composition. For example, when constructing bone modelsfrom Computerized Tomography (“CT”) and/or Magnetic Resonance (“MR”)images, bony structures need to be delineated from other structures(soft tissues, blood vessels, etc.) in the images. Also, each bone musttypically be separated from adjacent bones when modeling anatomicalstructures such as cervical spine or foot.

In 3D printing applications in the medical field, a useful feature isthe capability of building a prototype of a patient-specific anatomicalregion quickly. For example, if a patient comes in with a broken ankle,the surgeon may use a physical model of the bone fragments of thepatient to aid surgical planning, if the physical model can be generatedrapidly. For orthopedic surgeons, the ability to visualize andmanipulate a physical model of a bone or joint in need of repair priorto surgery can aid in the selection and design of surgical implants forfracture fixation or joint replacement. Rapid prototyping of patientspecific models increases efficiency and reduces costs by cuttingoperating room time. Rapid prototyping of patient specific models offerstremendous promise for improved pre-operative planning and preparation.While the technique of sizing surgical implants using newer imagingmodalities such as Computerized Tomography (“CT”) and/or MagneticResonance (“MR”) imaging is an improvement over standard X-ray films,the ability to work with an accurate physical model of the region ofinterest would produce further benefits, such as providing tactile 3Dfeedback of the relevant patient anatomy. Rapid prototyping or 3Dprinting refers to a collection of technologies for producing physicalparts directly from digital descriptions, which frequently are theoutput from Computer-Aided Design (CAD) software. Rapid prototypingmachines have been commercially available since the early 1990's, andthe most popular versions involve adding material to build the desiredstructure layer-by-layer based on a digital three dimensional model ofthe structure. For example, a physical model may be fabricated using arapid prototyping system using stereolithography, fused depositionmodeling, or three dimensional printing. In stereolithography, a laseris used to selectively cure successive surface layers in a vat ofphotopolymer. In fused deposition modeling, a thermal extrusion head isused to print molten material (typically a thermoplastic) that fusesonto the preceding layer. A typical three-dimensional printer uses aprinter head to selectively deposit binder onto the top layer of apowder bed.

SUMMARY

This summary is provided to introduce a selection of concepts in asimplified form that are further described below in the DetailedDescription. This summary is not intended to identify key features ofthe claimed subject matter, nor is it intended to be used as an aid indetermining the scope of the claimed subject matter.

The present application discloses systems and methods for single-actionprinting of 3D physical models from a three or n-dimensional image dataset. The methods may be applied to image data set obtained from any of awide variety of imaging modalities, including Computerized Tomography(“CT”), Magnetic Resonance (“MR”), positron emission tomography (“PET”),optical coherence tomography (“OCT”), ultrasonic imaging, X-ray imaging,sonar, radar including ground penetrating radar, acoustic imaging,microscopy imaging, simulated image data and the like, or combinationsof one or more imaging modalities. The systems and methods areapplicable to a wide range of applications from creating physical modelsof anatomical structures such as bones and organs to creating physicalmodels of mechanical components, archaeological sites, and naturalgeological formations.

The systems and methods described herein generally contemplate combiningprinting template methods with a 3D printer or rapid prototypingmachine. The printing template methods usually include predefined dataprocessing steps comprising identifying voxels in an image data set,generating a geometric representation, and sending the geometricrepresentation to the 3D printer to produce a 3D physical model. A 3Dprinter or rapid prototyping machine refers to a collection of devicescapable of producing three-dimensional physical parts directly fromdigital models using stereolithography, fused deposition modeling, threedimensional printing, sheet laminating or other technologies.

DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a current method of printing a 3D model from an imagedata set.

FIG. 2 illustrates a proposed method for printing a 3D model from animage data set.

FIG. 3 illustrates a flowchart of an exemplary single-action 3D modelprinting method.

FIG. 4 illustrates a flowchart of image data conversion steps includedin a printing template.

FIG. 5 illustrates a list of 3D printing templates accompanied anddescribed with text.

FIG. 6 illustrates a list of 3D printing templates accompanied anddescribed with text and graphics.

FIG. 7 illustrates 3D points as an exemplary geometric representationgenerated by a printing template from image data.

FIG. 8 illustrates a 3D contour as an exemplary geometric representationgenerated by a printing template from image data.

FIG. 9 illustrates a 3D triangle-based surface model as an exemplarygeometric representation generated by a printing template from imagedata.

FIG. 10 provides a table of sample CT numbers for various human tissues.

FIG. 11 illustrates an exemplary printing template of printing a bonestructure from a CT image data set.

FIG. 12 illustrates an exemplary printing template of printing a solidbody structure from an image data set.

FIG. 13 illustrates an exemplary printing template of printing aphysical model using predefined value ranges from an image data set.

FIG. 14 illustrates an exemplary process of printing a physical model ofselected organs or parts from an image data set.

FIG. 15 illustrates an exemplary image with seed voxels marked beforeimage segmentation.

FIG. 16 illustrates a segmentation result of a first round regiongrowing.

FIG. 17 illustrates a segmentation result of a second round regiongrowing.

FIG. 18 illustrates a segmentation result of a last round regiongrowing.

FIG. 19 illustrates a segmented image using a region growing technique.

FIG. 20 illustrates an example of user adjustable physical modelprinting method.

DETAILED DESCRIPTION

Certain specific details are set forth in the following description andfigures to provide a thorough understanding of various embodiments ofthe invention. Certain well-known details often associated withcomputing and software technology are not set forth in the followingdisclosure, however, to avoid unnecessarily obscuring the variousembodiments of the invention. Further, those of ordinary skill in therelevant art will understand that they can practice the inventionwithout one or more of the details described below. Finally, whilevarious methods are described with reference to steps and sequences inthe following disclosure, the description as such is for providing animplementation of embodiments of the invention, and the steps andsequences of steps should not be taken as required to practice thisinvention.

FIG. 2 illustrates an exemplary system using a single-action 3D printingmethod to print a 3D physical model from an input image data set 10. The3D model can be, for example, a patient-specific anatomical model.First, the image data set 10 such as CT data, MR data etc, is loaded ona computer 15. The image data set 10 is typically a voxel-based imagedata set depicting a 3D region with each voxel of the image data set 10encoding at least one image attribute, such as image intensity, color orthe like. A user 11 at the computer 15 selects one printing template 18from a list of printing templates (300 in FIG. 5) for printing a 3Dphysical model. The computer 15 applies the selected printing template18 to identify voxels in the image data, generate a geometricrepresentation in a data format supported by a 3D printer, and send thedata to a connected or networked 3D printer 30 for producing a physicalmodel 35. For example, the 3D printer 30 may comprise a rapidprototyping device as discussed above. The 3D printer 30 may beconnected to the computer directly through a local computer port, localarea network, or the Internet.

When a 3D printer 30 is not directly connected to the computer where aprinting template is used, the data generated from the printing templatemay be saved to a storage media (for example, a CD or DVD) or storagedevice (for example, a external hard drive). The saved data can then beported to the 3D printer 30 to generate the physical model 35.

FIG. 3 is a flowchart of a single-action 3D printing method. In FIG. 3,an image data set 10 is first received in step 210. In step 218, aselected printing template 18 is executed to identify the voxelcategories and generate a geometric representation for printing a 3Dmodel. In step 230, the generated geometric representation is sent to a3D printer 30 and in step 235, a 3D physical model 35 is produced.

In FIG. 3, step 218 represents a single user action involved in theprinting process of a 3D model. In step 218, selecting a printingtemplate includes a selection action by using a pointing device toposition on a specific printing template from a list of predefinedprinting templates and select the printing template for execution. Thesingle-action may be a clicking of a mouse button when a cursor ispositioned over a predefined area of a displayed list of printingtemplates or a depressing of a key on a key pad to select a specificprinting template.

A printing template as defined herein is a software program foridentifying voxels in an image data set, generating a geometricrepresentation of a 3D physical model in a data format supported by a 3Dprinter, and sending the geometric representation to a 3D printer tocreate a 3D physical model.

In general, 3D printers require a geometric representation of an objectin order to fabricate the geometric shapes required in making a 3Dphysical model. The geometric representation of an object may includeone or a combination of the following forms: a list of 3D points 501-506for the entire body of the object with locational and materialinformation defined at each 3D point (FIG. 7), a group of 3D contours552-561 to define the shape of the object on each image plane (FIG. 8),or surface models 580 (FIG. 9) consisting of triangles or polygons orsurface patches delineating the body of the object.

In the present application, a 3D physical model 35 may have one or morepieces and one or multiple colors, and may be made of one or multiplematerials. The conversion process from input image date set to ageometric representation understood by a 3D printer may be eitherdependent on or independent of imaging modality or any other imageinformation. A printing template may be implemented as a softwareprogram on a computer, a computer processing board, or the controllerboard of a 3D printer. It may be implemented as but not limited to: aprogram script file with processing instructions and parameters, abinary executable program with processing instructions and parameters, adynamically linked library (DLL), an application plug-in, or a printerdevice driver. A printing template may be implemented as a stand-alonesolution or a component of a system used for printing 3D physical modelfrom image data sets. A printing template program may be loaded locallyon a user's computer or reside on a remote server connected throughcomputer network.

FIG. 4 illustrates a flowchart of the image data conversion step in aprinting template. An input image data set 10 is received by a printingtemplate 15. The printing template starts its predefined voxelidentifying function 310 to identify voxel categories in the image data.After identifying the voxel categories, the printing template generatesone of three geometric representations (3D points 315, 3D contours 320or surface models 325) supported by a 3D printer and sends the generatedgeometric representation to the 3D printer 30 to produce a 3D physicalmodel 35.

The image data conversion process generates a geometric representationand any additional data needed for a 3D printer to print out thephysical model. The voxel identifying process is generally done usingimage processing techniques such as image segmentation andclassification. One purpose of image segmentation and classification isto identify the voxel categories at each voxel location for the entireimage data set. Commonly used image classification techniques includetrained classifiers (such as artificial neural networks), imageclustering using voxel similarity measures, etc. Commonly used imagesegmentation techniques include image thresholding, histogramthresholding, region growing, region splitting, watershed method, graphpartitioning, clustering, artificial neural network, and other methods.

The geometric representation generated from the input image data set for3D printing may be a list of 3D points 501-506 in the body of an objectwith locational and material information (FIG. 7) specified at eachpoint, or a set of 3D contours 551-561 to define the shape of an objectin the image planes (FIG. 8), or surface models of an object 580 (FIG.9), or a combination of them. The data generated for 3D printing is notlimited to geometrical representations such as points, contours orsurfaces as described. The data may also be organized as a list ofprinting instructions, such as “move to a location”, “deposit aspecified amount of building material”, “move to a new location”, etc.,that can be used to complete the physical model printing process.

The list of printing templates may be displayed as either text 300 (FIG.5) or graphics 301 (FIG. 6) on the computer 15. For example, the textmay use a description such as “bone structure” or “brain.” The graphicdisplay may use pre-drawn graphic icons to indicate “bone”, “skull”, or“brain.” The graphic display of a printing template may also use a 3Dgraphic rendering of the geometric representation generated from theimage data by the printing template. The number of printing templates isnot limited. Additional printing templates may be added for specificphysical model printing needs. New printing templates may be createdwith different processing steps and parameters. A printing template mayalso be implemented as a part of the input image; in which case userinteraction is not required. For example, when the input image isreceived with a specific printing template attached, the printingprocess starts automatically by executing the attached printingtemplate. The execution of the attached printing template may includesteps of first generating a geometric representation from the image andthen producing a physical model without any user interaction.

As described above, rapid prototyping systems build a physical model byadding consecutive layers, as opposed to subtractive rapid prototypingor conventional machining that uses a tool to remove material from blankstock. However, generation of a physical model may just as well useother processes and equipment. For example, rapid prototyping processesmay be adapted to produce functional objects (“parts”) rather than justgeometric models. In such case, rapid prototyping may be referred to bythe alternative names such as additive fabrication, layeredmanufacturing, and solid free form fabrication.

Many commercial rapid prototyping machines currently employ standardinput formats comprising of a polygonal representation of the boundaryof the object. For example, a CAD model or other three-dimensional(“3D”) digital model is converted to a list of triangles defining thesurface of the object. The machine slices through the collection oftriangles to generate a geometric representation that comprises theboundary of each layer to be printed or deposited. In the followingsections, different embodiments of 3D printing templates are discussed.

(1) Printing Bone Structure from CT Image Date Set

This embodiment is implemented as a printing template for printing aphysical model of a bone structure from a CT image data set.

In a CT image, the intensity value at each voxel may be converted to avalue in Hounsfield units (HU). The Hounsfield unit system measures theattenuation coefficient of tissues in computerized tomography.Hounsfield units are also termed CT numbers. FIG. 10 provides a table ofsample CT numbers for various human tissues. The table lists some of thevoxel values of different tissues or materials in Hounsfield units for atypical CT scanner. The values may differ on a different CT imagescanner due to specific settings on that particular imaging device andcustom calibrations of image data. The formula to calculate the CTnumber in Hounsfield units from the voxel intensity is normally providedas part of the image data. For example, the formula used by many CTscanner vendors is:

HU=Voxel Intensity*Scale+Intercept;

where HU is the voxel value in Hounsfield units, Voxel Intensity is theattribute value of each voxel provided in an image data set, and Scaleand Intercept are parameters provided with the formula. For example, formany CT images, Scale=1 and Intercept=−1000. Other values for Scale andIntercept may also be used.

As indicated in the table of FIG. 10, bone tissues may be identifiedusing a range of CT numbers (>1000). The value of every voxel in theimage data set can be checked to identify bone tissues. For example, ifa voxel has a value above 1000 HU, it is marked as bone tissue. Often anupper limit is used to prevent other hard materials such as metalimplants from being marked as bone tissue. A similar technique may beapplied to other tissues, such as soft tissue (fat, muscle, etc), blood,liver tissue, and white and grey matter in the brain.

FIG. 11 is a flowchart of an exemplary printing template for printingbone structures from a CT image set. The printing template identifiesvoxels that are part of the bone structure in the CT image 600,generates a geometric representation (in the format of 3D points 605,Contours 606, 606, or Surfaces 607) in a 3D printer supported dataformat and sends the geometric representation to the 3D printer 608 togenerate a physical model 610. The printing template includes thefollowing processing steps:

a) Go through the entire image data set 600 to check the HU value ofeach voxel (Step 601).b) For each voxel with a HU value larger than 1000 HU but less than anupper bound, mark the voxel with value 1 to indicate the voxel asrepresenting bone tissue (Step 602). Otherwise, mark it with value 0 toindicate non-bone tissue (Step 603). Repeat Step “a” and “b” until allvoxels are checked, in which case a geometric representation isgenerated (Step 604). The value 1000 HU is used here as an example.Different values or ranges may be used for different images.c) If the 3D printer (608) supports input data in the format of 3Dpoints, a geometric representation comprising a list of 3D points forall voxels marked with value 1 may be generated (Step 605) and sent tothe printer (Step 608) to generate a physical model 610. If otherinformation such as material or color is supported, we may include theother information in the geometric representation. FIG. 7 shows anexample of the 3D points generated from an image data set. In thisexample, every 3D point has an identification value which is either 0or 1. In this case, 0 indicates non-bone tissue and the voxels withvalue 0 are represented here by a white color. 1 indicates bone tissueand the voxels with value 1 are represented here by a dark color. Inother embodiments, every voxel may have one or multiple identificationvalues which may be any value, not limited to 0 or 1.

In FIG. 7, the list of 3D points are represented as:

Point 501: (5, 0, 0, 1) Point 502: (5, 1, 1, 1) Point 503: (4, 1, 1, 1)Point 504: (4, 2, 2, 1) Point 505: (5, 2, 2, 1) Point 506: (4, 3, 2, 1)

where each point has a data format of (X, Y, Z, Value). X, Y, Z are thethree-dimensional coordinates of a voxel and Value is the attribute witha value of, in this case, 1 for all the voxels identified as bone tissueand 0 otherwise. Other values may be used for identification purposes.Additional values may be also used to indicate attributes such as coloror material.d) If the 3D printer supports input data in the format of 3D contours, ageometric representation comprising the contours are generated bytracing along the outer edge of all voxels marked with the value of 1(Step 606). The contour tracing method is straight forward, and isnormally done by walking along the edge voxels in a fixed order withineach 2D image plane. For example, we may start the walk on an edge voxeland follow the next edge voxel in a clockwise fashion until the startingposition is encountered. The walking process is then repeated for allimage planes. FIG. 8 shows an example of tracing a contour in a 2D imageplane. In this example, the tracing process starts at one edge voxel 551and the contour starts with no point data. Voxel 551 is added to thecontour as the starting point. In a clockwise order, the next voxel onthe edge to be traced is voxel 552. Voxel 552 is added to the contour.Repeat the process to add voxels 553, 554, 555, 556, 557, 558, 559, 560and 561 to the contour. When the next edge voxel is the starting point(Voxel 551), the tracing process for this contour is complete. Thecontour may be represented as:

551: (4, 5, N, 1)—Start Point 552: (5, 5, N, 1) 553: (6, 5, N, 1) 554:(7, 5, N, 1) 555: (8, 4, N, 1) 556: (7, 3, N, 1) 557: (6, 3, N, 1) 558:(5, 3, N, 1) 559: (4, 3, N, 1) 560: (3, 3, N, 1) 561: (3, 4, N, 1)—EndPoint

where each point has a data format of (X, Y, Z, Value). X, Y, Z are thethree-dimensional coordinates of a voxel and Value is the attribute witha value of, in this case, 1 for all the voxels identified as bone tissueand 0 otherwise. Other values may be used for the attribute andadditional attributes such as color or material may be included as well.In this example, the particular tracing technique is described as anexample. Other tracing methods and variations may be used to generatesimilar results.e) If the 3D printer supports input data in the format of a surfacemodel, then a geometric representation in the format of a surface modelis generated using the “Marching Cubes” (U.S. Pat. No. 4,710,876,4,751,643, 4,868,748) method or other surface modeling methods (Step607). The generated geometric representation is sent to the 3D printerto produce a physical model 610 (Step 608). Most commercially available3D printers and rapid prototyping machines support the “STL” format,which stores surface geometry data as a set of raw unstructuredtriangles. For this example, the surface model 607 is sent to thethree-dimensional (“3D”) printer in the “STL” format.

“Marching cubes” is a computer graphics algorithm for extracting apolygonal mesh of an isosurface from three-dimensional voxels. Thealgorithm proceeds through the voxels marked with 1, taking eightneighbor locations at a time (thus forming an imaginary cube) and thendetermining the polygon(s) needed to represent the part of theisosurface that passes through this cube. The individual polygons arethen fused into the desired surface. The “Marching Cubes” algorithmgenerates triangle-based surface models. Additional post processingsteps such as surface smoothing and surface decimation may be applied toimprove the surface quality but are not required.

FIG. 9 shows an example of a three dimensional triangle-based surfacemodel 580. In this example, the triangle-based surface model has 8vertexes: P0, P1, P2, P3, P4, P5, P6, P7 and 12 surface triangles withT1, T2, T3, T4, T5, T6 displayed at the front of the model and T7, T8,T9, T10, T11, T12 displayed at the back of the model. Each vertex is a3D point: (X, Y, Z). Each triangle has 3 vertexes, for example (P0, P2,P1). This surface model may be represented as:

Triangle 1—T1, Front: (P0, P2, P1)

Triangle 2—T2, Front: (P1, P2, P3)

Triangle 3—T3, Front: (P2, P4, P3) Triangle 4—T4, Front: (P4, P2, P0)Triangle 5—T5, Front: (P4, P0, P5) Triangle 6—T6, Front: (P5, P0, P7)Triangle 7—T7, Back: (P6, P0, P7) Triangle 8—T8, Back: (P6, P1, P0)Triangle 9—T9, Back: (P6, P3, P1) Triangle 10—T10, Back: (P6, P4, P3)Triangle 11—T11, Back: (P6, P5, P4) Triangle 12—T12, Back: (P6, P7, P5)

In this example, the surface model representation is similar to thecommonly used “STL” format and may be sent to the 3D printer in the“STL” format for printing a physical model. Other representations andvariations, such as surface patches or polygon-based surfaces, may alsobe used.

The above example describes one embodiment of the single-action 3D imageprinting methods. The steps in the printing template may be combined orvaried. For example, the voxel checking and marking Steps “a” and “b”can be combined into Step “e” that checks the voxel values and generatesthe surface triangles without marking the voxels.

(2) Printing Solid Body Structure from an Image Data Set

This embodiment is implemented as a printing template for printing aphysical model of a solid body from an image data set.

For a known imaging modality, such as Computerized Tomography (“CT”) orMagnetic Resonance (“MR”) imaging, the voxels in an empty or no-tissueregion in an image typically have a known value range. For example, airwould be considered a no-tissue region. A voxel representing air has avalue range around −1000 HU as shown in the CT values table (FIG. 10).In other words, we can check the value of each voxel in the image dataset to identify whether the voxel represents an empty region or not. Forexample, if a voxel in a CT image has a value between −1000 HU and −200HU (the value below the lowest tissue value in Hounsfield unit), thevoxel may be identified as air. Otherwise the voxel may be identified asbody tissue. The same method may be applied to other imaging modalitiesto identify empty regions that are defined with known voxel valueranges.

FIG. 12 illustrates an exemplary printing template for printing a solidbody structure from an image data set. The printing template identifiesvoxels (Step 621) in empty regions and body regions in the image,generates a geometric representation (Steps 625, 626, or 627), and sendsthe geometric representation to the 3D printer to create a physicalmodel 629 (Step 628). The printing template includes the followingprocessing steps:

a) Go through the entire image data set 620 to check the value of eachvoxel in Hounsfield units (Step 621).b) For each voxel, if its value is within the value range of no-tissue(empty region, for example, air), mark the voxel with value 0 toindicate it is empty (Step 623). Otherwise, mark it with value 1 toindicate it has tissue (Step 622). Repeat Step “a” and “b” until allvoxels are checked, in which case, a geometric representation isgenerated (Step 624).c) If the output printing device supports input data in the format of 3Dpoints, we then generate a list of 3D points for all voxels marked withvalue 1 (Step 625) and send the list to the printer to generate aphysical model 629 (Step 628). If other information such as material orcolor is supported, we can extract such information from the input imagedata and send it together with the geometric representation in theformat of 3D points. See FIG. 7 for an example of the 3D pointsgenerated from an image data set.d) If the 3D printer supports input data in the format of contours, thena geometric representation may be generated by tracing the contoursalong the outer edge of all voxels marked with value 1 (Step 626). SeeFIG. 8 for an example of tracing a contour in a 2D image plane.e) If the 3D printer supports input data in the format of a surfacemodel, then a geometric representation in the format of a surface modelis generated (Step 627) using the “Marching Cubes” (U.S. Pat. Nos.4,710,876, 4,751,643, 4,868,748) method or other surface modelingmethods. The geometric representation is then sent to the 3D printer toproduce a physical model 629 (Step 628).

The above example describes one embodiment of the single-action 3D imageprinting methods. The steps in the printing template may be combined orvaried, for example, the voxel checking and marking Steps “a” and “b”can be combined into Step “e” that checks the voxel values and generatesthe surface triangles without marking the voxels.

(3) Printing Physical Model Using Predefined Voxel Value Ranges

This embodiment is implemented as a printing template for printing aphysical model from an image data set using predefined voxel valueranges.

FIG. 13 illustrates an exemplary printing template for printing aphysical model from an image data set 630 using one or more predefinedvoxel value ranges. A predefined value range may be in voxel intensity,color, texture, location, region, or any derived value from them. Atypical range has a low value and a high value to define the bounds ofthe range. A list of ranges may be used to define multiple value rangesthat are not adjacent to each other.

In this embodiment, the method identifies voxels using the predefinedvoxel value range (Step 631), generates a geometric representation(Steps 635, 636, or 637), and sends the data to the 3D printer togenerate a physical model 639 (Step 638). It includes the followingsteps:

a) Go through the entire image data set to check the value of each voxelagainst the value ranges defined in the printing template (Step 631).b) For each voxel with a value within the bound of one of the definedranges, mark the voxel with value 1 to indicate it is within thespecified range (Step 632). Otherwise, mark with value 0 to indicate itis outside (Step 633). Repeat Step “a” and “b” until all voxels arechecked and identified. A geometric representation is generated in Step634.c) If the 3D printer supports input data in the format ofthree-dimensional points, we generate a geometric representationcomprising a list of 3D points for all voxels marked with value 1 (Step635) and send the geometric representation to the 3D printer (Step 638)to create a physical model 639. If other information such as material orcolor are supported in the input image data 630, we may include theinformation and send it together with the geometric representation tothe printer. FIG. 7 shows an example of the 3D points generated from animage data set.d) If the 3D printer supports input data in the format of contours, ageometric representation can be generated by tracing the contours alongthe outer edge of all voxels marked with value 1 (Step 636). See FIG. 8for an example of tracing a contour in a 2D image plane.e) If the 3D printer supports input data in the format of a surfacemodel, a geometric representation may be generated using the “MarchingCubes” (U.S. Pat. Nos. 4,710,876, 4,751,643, 4,868,748) method or othersurface modeling methods (Step 637). The geometric representation isthen sent to the 3D printer (Step 638) to produce a physical model 639.Most commercially available 3D printers and rapid prototyping machinessupport the “STL” format, which stores surface geometry data as a set ofraw unstructured triangles. In a particular example, the surface model637 is sent to the 3D printer in the “STL” format.

The above example describes one embodiment of the single-action 3D imageprinting methods. The steps in the printing template may be combined orvaried. For example, the voxel checking and marking Steps “a” and “b”can be combined into Step “e” that checks the voxel values and generatesthe surface triangles without marking the voxels.

(4) Printing Physical Model of Selected Organs or Parts

This embodiment is implemented as a printing template for printing aphysical model of selected organs or parts from an image data set.

FIG. 14 illustrates an example of printing a physical model 649 ofselected organs or parts from an image set 640. To generated a physicalmodel of selected organs or parts, the image regions of the selectedorgans need to be identified using image segmentation techniques (Step644). A typical segmentation technique used for identifying specificimage regions starts with either a set of automatically generated (Step642) or user selected seed locations or regions (Step 641), grows eachregion by merging neighboring voxels that are within a certainsimilarity criterion, and repeats the process until no more neighboringvoxels are available for merging. For example, the criterion could be adifference of voxel intensity, gray level, texture, or color between thevoxels already identified and the ones being checked. After theidentification process is complete, the identified regions are then usedto generated a geometric representation for the 3D printer. Other imagesegmentation methods, such as region growing, active contours, graphpartitioning, watershed, and clustering, may be used in the image regionidentifying step of this embodiment.

In this example, the method identifies voxels using a region growingtechnique (Step 644), generates a geometric representation of theidentified voxels in a format supported by a 3D printer 648 (Steps 645,646, or 647) and sends the geometric representation to the 3D printer togenerate a physical model of selected organs or parts 649 (Step 658).

In this embodiment, a user 11 (referenced in FIG. 2) selects some voxelsor regions on the input image as seed voxels or regions. FIG. 15illustrates an example where two selected seed voxels 701 at location(4, 4, Z) and (5, 4, Z) are marked with dark color. The location isrepresented by the X, Y, Z coordinates of a voxel. FIG. 15 shows theoriginal image as a grayscale image. The grayscale value of this imageis from 0 to 255. Seed voxels are checked (Step 641) to see whether theyare available and whether they are within the bound of the image set. Ifseed voxels are available, continue to Step “b” to start the imagesegmentation process (Step 644) through region growing. If seed voxelsare not available, Step 642 is carried out to generate seed voxelsautomatically. For example, a predefined value range may be used toselect seed voxels within a value range.

b) The image segmentation process through region growing starts at theseed voxel locations. The process grows each region by mergingneighboring voxels that are within a certain similarity criterion andrepeats the process until no more neighboring voxels are available formerging. In this example, we choose a simple criterion for measuringvoxel similarity, that is, for a neighboring voxel to qualify as similarto a reference voxel, the grayscale value difference between theneighboring voxel and the reference voxel must be less than 20. In otherwords, if the grayscale value difference is less than 20, theneighboring voxel is added to the region and the region grows by onevoxel. If the grayscale value difference is equal to or greater than 20,the voxel is not added to the region. For example, voxel 702 (4, 3, Z)in FIG. 15 has a grayscale value of 103. The difference between thisvalue and the grayscale value (102) of the voxel located at (4, 4, Z)701 is 1. The difference is within the similarity criterion, so theneighboring voxel 702 is added to the region. FIG. 16 illustrates thefirst round of region growing for the image example shown in FIG. 15.All neighboring voxels with grayscale difference less than 20 are markedwith a dark color. 8 voxels 702 (marked with horizontal hatch lines) atlocations (4, 3, Z), (5, 3, Z), (3, 4, Z), (6, 4, Z), (3, 5, Z), (4, 5,Z), (5, 5, Z), (6, 5, Z) are added to the region. Here the image planeis assumed to be parallel to the XY plane therefore all voxels have thesame Z value.

FIG. 17 illustrates the result of the second round region growing. 3voxels 703 (marked with vertical hatch lines) at locations (4, 6, Z),(5, 6, Z), (6, 6, Z) are added to the region.

FIG. 18 illustrates the result of the last round region growing. 1 voxel704 (marked with diagonal hatch lines) at locations (5, 7, Z) is addedto the region. After this round, no more voxels meet the similaritycriteria. The region growing process stops. All voxels added to theregion are marked with value 1 and the rest of the voxels are markedwith value 0 as shown in FIG. 19.

In FIG. 14, after the image region of the selected organ has beenidentified (Step 644), a geometric representation is generated in theformat of either 3D points, or contours, or surfaces (Step 645, 646, or647). The geometric representation is then sent to a 3D printer tocreate a physical model 649 (Step 648).

The above example describes one embodiment of the single-action 3D imageprinting methods in which the image segmentation technique uses a regiongrowing method. Other image segmentation methods may be used to generatesimilar results.

(5) User Adjustable Physical Model Printing Method

This embodiment is implemented as a printing template for printing aphysical model from an image data set using user adjustable imageprocessing parameters and steps.

FIG. 20 illustrates an exemplary printing template that adopts a useradjustable physical model printing method. This printing templateprovides a user with a list of selectable processing options andadjustable parameters for image segmentation and data conversion. Theuser 11 makes the initial selection of parameters and segmentationmethods (Step 651). The segmentation methods may include the onesdescribed in the above examples, such as region growing, imagethresholding, graph partitioning, and others. The parameters may includevalue ranges that are adjustable, user defined seed regions, and others.The image segmentation process (Step 654) segments the input image usingthe selected methods and parameters. The geometric representationgenerated from the image segmentation process is used to create a 3Drendering (Step 660) to show how a final physical model may look on acomputer. The computer 15 (FIG. 2) can be used to display athree-dimensional rendering 660 of the model generated by the imagesegmentation process. The three-dimensional rendering may be implementedusing volume rendering of the segmented image or surface rendering fromthe surface model 657. If the rendering meets the user's requirements,the segmented image is converted to a format supported by the 3D printer658 for printing (Steps 655, 656, or 657). Otherwise, the user can makeadditional adjustments. The user can decide to print the 3D model on a3D printer using the current settings or continue the adjustment (Step651) until the user is satisfied with the settings for printing.

Similar to other embodiments as described above, this embodiment furtherincludes the processing steps of converting an image data set 650 to ageometric representation in a data format supported by the 3D printerand sending the geometric representation to the 3D printer to generate aphysical model of selected organs or parts 659 (Step 658).

(6) Extension to n-Dimension

Although the present method has been described with reference to 3Dimage data sets, it will be immediately apparent to persons of skill inthe art that the methods described above are readily applicable to anynumber of dimensions. It is contemplated that the methods may be appliedto n-dimensional data, where n may be 2, 3, 4 or any number larger than4. In particular, it is contemplated that the invention may be appliedto n-dimensional data in which one of the dimensions is time and thereare two or three spatial dimensions. For example, we can use the abovedescribed methods to produce multi-dimensional physical models thatevolve over time.

It should be appreciated that the present method greatly reduces thetime required for printing physical models from an n-dimensional dataset, including a 3D data set. Therefore practical applications capableof producing a series of physical models from time-sequence image datasets to show changes of shape or motions can be implemented. Forexample, time-sequence 3D image data of a chest containing a beatingheart may be used to generate a series of chest models to show the shapeand motion of the beating heart at different time points.

Although the present invention has been described in terms of variousembodiments, the invention is not limited to these embodiments.Modification within the spirit of the invention will be apparent tothose skilled in the art. For example, various different single-actionscan be used to effect the printing of a physical model from an imagedata set. For example, a voice command may be spoken by the user. A keymay be depressed by the user. A button on a 3D printing device may bepressed by the user. Selection using any pointing device may be effectedby a user to start the execution of a printing template. Although asingle-action may be preceded by multiple physical movements of the user(e.g., moving a mouse so that a mouse pointer is over a button), thesingle-action generally refers to a single event received by a systemthat commands the system to print a physical model from an image dataset or a derived representation of the image. Finally, varioustechniques for identifying voxel categories and generating a geometricrepresentation can be used to print a physical model from an image dataset.

What is claimed is:
 1. A printing template method of single-action 3Dprinting of a solid body comprising steps of: receiving an image dataset as input; identifying voxels in empty regions and in regions withtissues or materials; generating a geometric representation from theidentified voxels; and sending the generated geometric representation toa 3D printer for printing a physical model.
 2. The method of claim 1,wherein the image data set has a plurality of voxels, each voxel havingan intensity value; wherein the image data set comprises images of oneor more three-dimensional regions obtained using one or more imagingmodalities selected from magnetic resonance imaging, computedtomography, ultrasound, X-ray imaging, positron emission tomography,microscopy imaging, simulated image.
 3. The method of claim 1, whereinthe identifying of voxels includes checking the image attribute of eachvoxel against a predefined criteria for the empty regions to identifywhether the voxel is in an empty region or not.
 4. The method of claim3, wherein the predefined criteria may be one or more value ranges toidentify whether the voxel is within the ranges or outside the ranges.5. The method of claim 1, wherein the identifying of voxels includeschecking the image attribute of each voxel against a value rangeindicating no-tissue to identify no-tissue or empty regions; whereinvoxels that are outside the range are identified as tissue regions. 6.The method of claim 1, wherein the geometric representation may be alist of 3D points, a list of 3D contours, a list of 3D polygonal surfacemodels, or a combination of them.